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DETAILED ACTION 

Response to Amendment 

Applicant's response received on 3/4/2008 is fully considered herein. Claims 1- 
22 are currently pending. 



Claim Rejections - 35 USC §102 

1 . The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(a) the invention was known or used by others in this country, or patented or described in a printed 
publication in this or a foreign country, before the invention thereof by the applicant for a patent. 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 



(e) the invention was described in (1 ) an application for patent, published under section 1 22(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 



2. Claims 1, 11, 14 and 16 are rejected under 35 U.S.C. 102(e) as being anticipated 
by Gong et al. (US 7,016,540). 

Regarding claims 1 and 14, Gong discloses [a] key-frame extraction system 
("The invention also seeks to fulfill different user requirements for video browsing and 
content overview by outputting either an optimal set of keyframes representing the input 
video sequence", Gong, column 5, line 47), comprising a set of frame analyzers that 
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each select a set of candidate key-frames from among a series of video frames in a 
video, each frame analyzer for detecting a meaningful content in the video("Performing 
subsequent singular value decomposition on this matrix A projects each frame i from 
the m-dimensional raw feature space into a K-dimensional refined feature space 
(usually K«m, though this is not required). In this new feature space, noise and trivial 
variations in video frames are ignored, and frames with similar color distribution patterns 
are mapped near to each other", Gong, column 6, line 1) such that the candidate key- 
frames comprise a subset of the video frames in the video ("Other aspects of the 
inventive method include arranging the selected frames into a feature frame matrix, and 
performing the singular value decomposition on this feature frame matrix", Gong, 
column 5, line 7); 

key-frame selector that arranges the candidate key-frames into a set of clusters 
("Therefore, the K-dimensional vectors representing each of the frames in the refined 
feature space can be used not only for clustering visually similar frames for content 
summarization, but also for accurately segmenting video frames into shots, and also for 
similarity matching among the detected shots", Gong, column 6, line 8). 

Regarding claim 11 and 16, Gong The key-frame extraction system of claim 14, 
wherein the frame analyzers include a color histogram analyzer ("The video frames of 
the input video sequence are represented in the method according to an embodiment of 
the present invention using color histograms", Gong, column 6, line 55). 
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Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. Claims 2, 4, 5, 7-10, 18, 19 and 21 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Gong et al. (US 7,016,540) in combination with Li et al. (US 
2003/0210886). 

Regarding claims 2 and 19, Gong discloses [t]he key-frame extraction system of 
claim 14. 

Gong does not disclose wherein the frame analyzers include a camera motion 
tracker. 

Li working in the same field of endeavor of key-frame extraction wherein the 
frame analyzers include a camera motion tracker ("Camera motion may be detected by 
analyzing the layout pattern of the extracted motion vectors mv", Li, Paragraph 42 and 
figure 3). 
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It would have been obvious at the time the invention was made for one of 
ordinary skill in the art to include the camera motion tracker of Li in the key-frame 
extraction system of Gong because "when the underlying video content of a scene 22 is 
highly dynamic (usually characterized by lots of camera motion, many object activities 
and a large number of contained shots), the scene 22 has complex contents, and thus 
deserves more keyframes" (Li, column 33). 

Regarding claims 4 and 18, Gong teaches [t]he method of claim 1. 

Gong does not teach wherein selecting a set of candidate key-frames includes 
selecting a set of candidate key-frames in response to a fast camera movement in the 
video. 

Li, working in the same field of endeavor of key-frame extraction does teach 
wherein selecting a set of candidate key-frames includes selecting a set of candidate 
key-frames in response to a fast camera movement in the video ("The last two factors 
are used to make sure that the extracted keyframe is a well-focused clear image and 
not a blurry image, such as is caused by a fast camera motion, fast object movement or 
a bad camera focus. For example, the still image taken after a camera panning is 
preferred over the image taken during the panning which may be blurred or unstable", Li 
paragraph 52). 



Application/Control Number: 10/807,949 
Art Unit: 2624 



Page 6 



It would have been obvious at the time the invention was made for one of 
ordinary skill in the art to use the fast motion detection teachings of Li in the key-frame 
extraction system of Gong since fast camera motion causes blurring and it is better to 
avoid those frames and to use frames outside a sequence of fast camera motion. 

Regarding claims 5 and 21 , Gong discloses [t]he key-frame extraction system of 
claim 14. 

Gong does not disclose wherein the frame analyzers include a human face 
detector. 

Li does teach wherein the frame analyzers include a human face detector ("Four 
factors are considered in determining the importance of a frame 26 according to one 
embodiment of the invention: 1) 2) the number of detected human faces in the frame 
Li, paragraph 52). 

It would have been obvious at the time the invention was made for one of 
ordinary skill in the art to use the face detector frame analyzer as taught by Li in the 
key-frame extraction system of Gong help judge the importance of a frame as "generally 
speaking, a frame 26 that contains a human face will be more informative than, for 
example, a landscape frame", Li, paragraph 52). 



Regarding claim 7, Gong teaches [t]he method of claim 1 . 
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Gong does not teach wherein selecting one of the key-frames from each cluster 
includes determining an importance score for each candidate key-frame. 

Li does disclose wherein selecting one of the key-frames from each cluster 
includes determining an importance score for each candidate key-frame ("an 
importance value is assigned to each scene, shot and frame", Li, paragraph 8). 

It would have been obvious at the time the invention was made for one of 
ordinary skill in the art to use the importance score of Li in frame analysis of the key- 
frame extraction system of Gong as "different shots may be of different importance to 
users, it is preferably to assign more keyframes to important shots than to the less 
important ones" (Li, paragraph 6). 

Regarding claim 8, Gong discloses [t]he method of claim 7. 

Gong does not disclose wherein determining an importance score for each 
candidate key-frame includes determining an importance score in response to the 
meaningful content in each candidate key-frame. 

Li does teach wherein determining an importance score for each candidate key- 
frame includes determining an importance score in response to the meaningful content 
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in each candidate key-frame ("FIG. 1c is a flowchart illustrating one embodiment of the 
computation of importance values according to the invention", Li, paragraph 1 1 , and 
figure 1c). 

It would have been obvious at the time the invention was made for one of 
ordinary skill in the art to use the importance score of Li in frame analysis of the key- 
frame extraction system of Gong as "different shots may be of different importance to 
users, it is preferably to assign more keyframes to important shots than to the less 
important ones" (Li, paragraph 6). 

Regarding claim 9, Gong teaches [t]he method of claim 1 . 

Gong does not teach wherein selecting one of the key-frames from each cluster 
includes selecting one of the key-frames in response to an image quality of each 
candidate key-frame. 

Li does teach wherein selecting one of the key-frames from each cluster includes 
selecting one of the key-frames in response to an image quality of each candidate key- 
frame ("The last two factors are used to make sure that the extracted keyframe is a well- 
focused clear image and not a blurry image", Li, paragraph 52). 
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It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to use image quality as a means of selecting key-frames as taught by Li in 
the key-frame extraction system of Gong since blurry or out-of-focus images will be less 
aesthetically appealing to a viewer. 

Regarding claim 10, Gong teaches [t]he method of claim 1 . 

Gong does not teach further comprising selecting multiple key-frames from each 
cluster and obtaining a user selection for the multiple key- frames. 

Li does teach further comprising selecting multiple key-frames from each cluster 
and obtaining a user selection for the multiple key- frames ("if the user wants a detailed 
summary of certain scenes 22 or shots 24 while only wanting a brief review of others, 
the invention as described herein can easily achieve this by using a predefined but 
tunable scale factor", Li paragraph 73, the system selects an initial key-frame set and 
the user can control how many to select based on a tunable scaling factor). 

It would have been obvious at the time the invention was made for one of 
ordinary skill in the art to include the user interaction taught by Li in the key-frame 
extraction system of Gong since "sometimes the user may want to take a detailed look 
at certain scenes or shots which requires more keyframes, and sometimes the user may 
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only need a very coarse summarization which requires fewer keyframes" (Li, paragraph 
6). 



5. Claims 3, 6, 13, 15, 20 and 22 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Gong et al. (US 7,016,540) in combination with Chang et al (US 
2004/0125877). 

Regarding claims 3 and 20, Gong discloses all the elements of claims 1 and 14 
above. Gong does not explicitly teach selecting a set of candidate key-frames in 
response to an object motion in the video. 

Chang, working in the same field of endeavor of video analysis, does teach 
selecting a set of candidate key-frames in response to an object motion in the video ("a 
Visual Feature Extraction Module 340 extracts visual features that can be used for view 
recognition or event detection. Examples of visual features include camera motions, 
object motions, color, edge, etc.", Chang, paragraph 104). 



It would have been obvious at the time the invention was made for one of 
ordinary skill in the art to add the object motion detection feature of Chang to the video 
analysis system of Gong in order to aid in detecting scene changes, "Since there are 
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many different changes in video (e.g. object motion, lighting change and camera 
motion), it is a nontrivial task to detect scene changes", (Chang, paragraph 77). 

Regarding claims 6 and 15, Gong does not disclose an audio event detector that 
selects a set of candidate key-frames by detecting a set of audio events in the video. 

Chang, working in the same field of endeavor of video analysis, does teach an 
audio event detector that selects a set of candidate key-frames by detecting a set of 
audio events in the video ("An Audio Feature Extraction module 345 extracts audio 
features that are used in later stages such as event detection", Chang, paragraph 105). 

It would have been obvious at the time the invention was made for one of ordinary 
skill in the art to add the Audio Feature Extraction Module feature of Chang to the video 
analysis system of Gong in order to aid in detecting scene changes that have 
characteristic sound events associated with them ("more detailed analysis of motion 
such as speed, direction, repeating patterns in combination with audio analysis (e.g., 
hitting sound) may be needed.", Chang, paragraph 185). 

Regarding claims 13 and 22, Gong does not disclose a user interface for displaying 
a set of video frames in the video previous to each key-frame and a set of video frames 
in the video subsequent to each key-frame and for obtaining a user selection of one or 
more of the video frames. 
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Chang, working in the same field of endeavor of video analysis, does teach a user 
interface for displaying a set of video frames in the video previous to each key-frame 
and a set of video frames in the video subsequent to each key-frame and for obtaining a 
user selection of one or more of the video frames ("there may be false alarms or misses 
associated with the indexing process. A browsing interface may be used for users to 
identify and correct false alarms. For errors of missing correct scene changes, users 
may use the interactive interface during real-time playback of video to add scene 
changes to the results", Chang, paragraph 100). 

It would have been obvious at the time the invention was made for one of 
ordinary skill in the art to add an interactive interface as taught by Chang to the video 
analysis system of Gong so "If a user is monitoring the scene change detection process 
and notices a miss or false detection, he or she can hit a key or click mouse to insert or 
remove a scene change in real time." (Chang, paragraph 99). 

6. Claims 12 and 17 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Gong et al. (US 7,01 6,540) in combination with Dufaux (US 6,71 1 ,587 B1 ). 

Regarding claims 12 and 17, Gong discloses all the elements of claims 1 and 14. 
Gong does not teach wherein the frame analyzers include a color layout analyzer. 
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Dufaux, working in the same field of endeavor of keyframe selection does teach 
wherein the frame analyzers include a color layout analyzer ("Returning to FIG. 5, at 
step 502 a pixel-wise frame difference number is calculated for each frame. A measure 
of the amount of difference between pixels in successive frames may be used to 
determine a shot boundary in the digital video file", Dufaux, column 8, line 37, Dufaux's 
method computes color differences at specific x,y locations corresponding to a physical 
layout of the features). 

It would have been obvious at the time the invention was made for one of 
ordinary skill in the art to use the frame difference method of Dufaux with the video 
analysis system of Gong to provide another method to test for shot boundaries since "A 
high value of pixel-wise frame difference indicates a possible shot boundary" (Dufaux, 
column 8, line 47). 

Response to Arguments 

7. Applicant's arguments with respect to claims 1-22 have been considered but are 
moot in view of the new ground(s) of rejection. 
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Conclusion 

8. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to THOMAS M. REDDING whose telephone number is 
(571)270-1579. The examiner can normally be reached on Mon - Fri 7:30 am - 5:00 pm 
EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Vikkram Bali can be reached on (571) 272-7415. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

IT. M. R./ 

Examiner, Art Unit 2624 



A/ikkram Bali/ 

Supervisory Patent Examiner, Art Unit 2624 



